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ABSTRACT 

Designed to improv«2 uhder^^chievihg high school 

students' Reading atid cbimhuh i cat ion skills , the Secondary 
Developmental Reading program served 909 ninth, tenth, and eleventh 
graders in 13 Ohio schools. Fuading was made available through the 
Ohio Disadvantaged Pupil Program Fund (DPPF),_Nine teachers from 
eight high schools participated in the 1985-1986 program, using Apple 
computers as instruction aids, and diagnostic tests to assess 
students 'individual reading strengths and weaknesses. Data analyzed 
included pretest and posttest scores, and inservice evaluation forms. 
Specific objectives of the program were (Ij that pupils who attended 
80% of the 7-month training period would show ah average of 1.0 
Normal Curve Equivalency (NCE) for each months and (2) that program 
personnel would be provided at least two inservice sessions and that 
at least 89% of the personnel attending each session would rate the 
session as valuable^ Neither goal was attained. Cost per pupil was 
greater and NCE gains smaller in the computer-assisted group than in 
the regular group. One of the inservice sessions was rated as 
valuable by^less than the requisite 80%. An extensive list of program 
recommendations was complied, including ways to increase attendance 
and to review the program to determine whether it should be 
continued, modified, or discontinued, (Tables of data are included, 
and evaluation forms are appended.) (AEW) 
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_ . FINAL EVALUATION FLEPORT 
LANGUAGE DEVELOPMENT COMPONENT 
SECONDARY DEVELOPMENrAL READING 
1985-86 

ABSTRACT 



Pro^^am- Description: The Secondary Developmen tal Reading (SDR) Program served 
909 pupils in grades 9-11 (only one pupil at grade 11) in 13 senior high 
schools. Funding of the compplnejit was made available through the Ohio 
Disadvantaged Pupil Program Fund (DPPF), 

The purpose of the SDR Program is to assist underachieving senior high 
pupils in raising their reading and communication skills. Emphasis of the 
program is placed on literacy survival skills necessary to function in our 
word-oriented world. 



Within the 1985-86 SDR Program nine _ teachers in eight senior high schools 
participated in a project which utilized Apple computers for computer assisted 
instruction/computer management system /CAI /CMS) . The computers, software^ and 
attendant serviceis were contracted v;it: the Prescription Learning (PL) Company 
9^_SP^i'^Sf ield, Illinois. The regular treatment group had six teachers in five 
senior high schools. 

Time Xnterv^^ For evaluation purposes^ the Secondary Developmental Reading 
Program started on September 16i 1 985 _ and continued through April 18, 1986. 
This interval of time gave 134 possible days of program instruction* Pupils 
included in the final pretest-posttest analysis must have attended at least 107 
days (80%) during the time period stated above* 

Ac£i£lties£ The program made use of diagnostic testing to assess pupils' 
individual reading strengths and weakness. Individualized instruction to meet 
pupils' needs was provided on a daily basis ii a small group setting. 



Program- xyh-j^ t ives^^ The program had twD objectives. Objective 1.1 stated that 
^"^^^^^^i^*^ Scimple will be comprised of pupils who score at or below the 
36%ile on a selection test and are in attendance at least 80% of the 
instructional period. Pupils who attend 80% of the. seven month treatment 
period will show an average gain in reading of 1.0 NCE for each month, which is 
an average gain of 7.0 NCE's overall (seven, months, x 1.0 NCE). Objective 2.1 
stated that program personnel will be provided at least two inservice sessions 
and that at least 80% of the personnel attending each session will rate the 
session as yaluable in providing information that will assist them in carrying 
out their program responsibilities. 

Evaiaa£± oii De si gn: Objective 1.1 was evaluated through jthe administration of 
the Comprehens i ve Tests- Ba sic S kills (CTBS) Reading Comprehension subtest, 
^aiyse^ of the data included comparison of pretest to posttest change scores 
in terms of grade equivalents, percentiles, and NCE's. Objective 2.1 was 
evaluated by means of the General Inservice Evaluation Form, a locally 
constructed instrument. 

Ma Jox- 1 i nd i ng s /Re comme n dat^lons-^ The information collected on the Pupil Census 
Forms indicated the program served 909 pupils for an average of 3.6 hours of 
instruction per week. The average daily membership in the program was 799.6 
pupils. The average days of enrollment per pupil * was 117.9 days and the 
average attendance per pupil was 99,8 days. The average number of pupils 
served per teacher was 53.3. 
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Objective 1.1, that pupiis_ who attended 80% of the seven month treatment 
period would show an average gain in reading of 1.0 NCE for each months was not 
attained.. There was a negative average change of -8.5 or -1.2 NCE/morith. It 
was postulated that three factors jprescheduiing of classes for SDR^ outdated 
selection test scores^ and problems in the administration of the customized 
posttest, could have contributed to the poor results of the SDR program. 

_ Objective 2.1, that program personnei would be provided at least two 
iriservice sessions and that at least 80% of the personnel attending each 
session would rate the session as valuable in providing information to assist 
them in carrying out. their program responsibilities, was technically not 
attained because one of the five sessions was rated as valuable in carrying out 
component: responsibilities by less than the requisite 80% of the participants. 
Each of the four remaining sessions, however, was favorably rated by well over 
80% of the participants. When a coiabined rating of all five insefvice sessions 
were computed^ an overall average_ of 90. 2% of the participants rated the 
inservice sessions as valuable in carrying out component responsibilities. 

The CAiyCMS project was located in eight high schools. The computer 
assisted units served 569 pupilsi Neither the CAI/CMS project group nor the 
group receiving regular program^ instruction attained the achievement 
criterion. The CAI/CMS group had a negative change of -9.8 NCE's in a seven 
month period, while the regular group _had a negative change of -6.1 NCE's. The 
achievement criterion was met in grade 10 of the regular group, with an average 
gain of 7.8 NCE's for the treatment period, or i.i NCE's per month. 

A cbst-benef it study indicated . that cost per pupil was greater and NCE 
gains smaller. in the CAI/CMS group than in thr regular group. Based on average 
daily membership, the cost per pupii_ was $989.38 in the CAI/CMS group and 
$704.81 in the regular group. Differences in NCE gains were noted above. 
However, CAI/CMS teachers served an average of 5.^3 more pupils per teacher than 
in the regular group, based on average daily membership, and attendance was 
somewhat better in the CAI/CMS groups 

The following program recommendatiojis were made : (a) the program should be 
conducted in schools that will work with program personnel to reduce scheduling 
problems arid increase program at tendance ;_(b) the selection of pupils for the 
program should be based on the most current test data; (c) the professional 
judgment of classroom teachers should be given considerable weight in the 
selection process^ and a_ system should be devised_ for obtaining teacher 
recommendations from the feeder middle schools; (d) the CAI/CMS part of the 
program should continue to be evaluated with an _e ye towards finding more 
effective methods of serving the high school pupil i^o is experiencing reading 
problems; (e) selection procedures, correlation of course content to system's 
Course of Study, instructional methods, class jsize, and test content should be 
reviewed to determine _ why pupils are not showing desired growth; (f) school 
administrators arid staff should take the responsibility of assuring an optimum 
testing erivironmerit by not scheduling unsuitable activities during testing 
weeks and adjusting class schedules to accommodate the length of the tests; (g) 
conditions for the pretest, and posttest should be as comparable as possible; 
and (h) the program should be extensively reviewed to determine whether the 
program model should be continued in its present form, modified, or 
discontinued. 
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Program Descri7£ian 



^ The Secondary Developmental Reading (SDR) Program began in the Columbus 
Public Schools in the fall of 1971 as a component of the Ohio Disadvantaged 
Pupil Program Fund. The 1985-86 version of the SDR Program was located in 13 
Columbus senior high schbol buildings. Fifteen project reading teachers worked 
in these 13 schools with 909 pupils in grades 9-11 who scored at or below the 
36th percentile on a standardized achievement test in reading. 

Within the 1985-86 SDR Program nine teachers J.n eight senior high schools 

participated in a project which utilized Apple computers for computer assisted 
instruction/computer management system (CAl/CMS). The computers, software, and 
attendant services were contracted with the Prescription Learning (PL) Company 
of Springfield, Illinois, In addition to providing a new technique to reading 
and language instructibri, the use of CAI/CMS was intended to enable teachers to 
serve more pupils than would be possible in regala? SDR classrooms. The use of 
CAI/CMS was also intended to be a cost-effective alternative to replacing badly 
worn conventional equipment. Of the 909 pupils in the SDR program, 569 
received computer assisted instruction. 

The purpose of the SDR Program is to assist underachieving senior high 

pupils in raising their reading and communication skills. Emphasis of the 
program is placed on literacy survival skills necessary to function in our 
word-oriented world. 



Features of the SDR Program include the following: 



1. 


Diagnostic testing to assess a pupil's 
reading strengths and weaknesses. 


individual 


2. 


Individualized instruction tailored to 


meet the needs of pupils. 


3. 


Small group instruction. 




4. 


On-going evaluation of pujjils to assess 


their reading needs. 


5. 


Inservice meetings for teachers. 
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Evalu^tlvon Objectives 

Objective 1.1 An evaluation sample will be comprised of pupils who score at or 
below the 36%tle_on a selection test and are in attendance at least 80% of the 
instructional period. The average reading growth of pupils In the evaluation 
sample and participants in the Computer Assisted Instruct Ion /Computer 
Management System (eAi/CMS) id;ll be 1.0 normal curve equivalent (NCE) point for 
each month of instruction. 

Objective 2.1 To provide at least two inservice sessions to program personnel 
such that at least 80% of the inservice participants will rate each session as 
valuable in providing information that will assist them in carrying out their 
program responsibilities. 



Evalua t i on— De si gn 

The evaluation design for the SDR Program called for the collection of data 
in three areas. 



1. Pupil Census Information 

The_ Pupil Census Forra_ TOs developed for the purpose of 
collecting pupil demographic and participation data in the 
Secondary Developmental Reading Program (SDR). Project 
teachers maintained the Pupil Census Forms for all pupils 
throughout the school year or when the pupils left the 
program. Data collected on the Pupil Census Forms were the 
number of days the pupil was enrolled in the program, the 
number of days the pupil was in attendance, and the average, 
number of tours per week the project teacher served the 
pupil. Other information collected included the pupil's grade 
and sex^ identification of non-English speaking pupils^ 
idehtif ication of any pupil v^o left the DPPF program because 
of qualifying for a special education program, and a question 
regarding a pupil's progress which required a subjec:ive 
resj^onse from the proje( t teacher, A copy of the Pupil Census 
Form can be found in the Appendix. 

Standardized Achievement Test information 

The purpose of the administration of the standardized 
achievement test was to collect pretest-post-.est achievement 
data on all SDR Program pupils to determine If Objective 1.1 
was achieved. Thc^' standard achievement test used was the 
Comprehensive Tests of Basic Skills (CTBS), Reading 
Cbmprehensibn (CTB-McGraw Hiii, 1981). The CTBS Reading 
Comprehension tests were administered on September 30 - 
October 4^ 1985, and again on April 21-30, 1986. The 
following lists the form, subtest and test levels of the CTBS 
used for each grade level. 
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Grade 



Snbtest 



Pbsttest 
Level J Form V* 
Level J Form V 



9 Reading Comprehension Level J Form U 

10-11 Reading Comprehension Level J Form U 

*Estimated by administration of abridged Form V, 

At pbsttest time, grade nine was administered a customized test that 
provided norm-referenced as well as criterion-referenced scores. The 
customized tests were developed by Columbus Public Schools personnel in 
cooperation with CTB/McGraw Hill to match the Columbus Public Schools 
Graded Course of Study i 



The achievement tests were administered as follow: P'-ogram teachers in 
grades 9-10 normally administered the pretest except in schools where 

schoolwlde testing occurred. Posttests for grade 9 were administered 

as part of districtwide testing^ Grade 10 was one of the exceptions to 

districtwide testing, and teachers of grade 10 pupils had to administer 
their own posttests. ^^i^8 schooiwide or districtwide tissting, tests 
were administered by classroom teachers with program teachers serving 
as proctors in sorae classrooms. Pretesting occurred during the week of 
September 30 - October 4 1985; posttesting occurred April 21-30, 1986. 

3. Inservlre Evaluation 

The locally developed General inservice Evaluation Form was designed to 
obtain teacher perceptions regarding each inservice session. The form 
was administered to participants at the close of inservice sessions. A 
modified version of the form was used for the orientation meeting of 
September 3, 1985, which was attended by regular SDR and CAI/CMS 
teachers. There was a total of five inservice meetings - three of 
VThich were available to regular SDR teachers and four of iwhich were 
available to SDR teachers in the CAl/CMS project. The dates and topics 
of inservice sessions in the 1985-86 school year were as follows: 



September 3, 1985 

November 22, 1985 

December 4, 1985 

January 23, 1986 
March 12, 1986 



Opening Conference (All SDR teachers - 
ail day program) 

Motivational Strategies for the 

classroom (secondary CAI/CMS teachers) 



Instructional Planning (SDR teachers 
in regular treatment group) 

The Writing Process (all SDR teachers) 



Bank Street Writer and Time-on Word 
Processing Programs (Secondary 
CAI/CMS teachers) 
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Teachers completed, inservice evaluation forms for all of the above 

???tings. A copy of the General Inservice Evaiuation Form and a copy of the 
modified version used in the orientation meeting are found in the appendix. 

addition to the types of data specified in the evaluation design* 

process evaluation data were obtained in a series of on-site visits to regular 
SDR and CAI/CMS program classrooms. Observations were conducted by persbrihel 
from the Department of Evaluation Services during the period from February 5 
through March 20, 1986. The purpose of these observations was to obtain teacher 
input regarding the program's functioning. Observations were conducted by a 
project evaiuator to the eight high schools having eAI/CMS units, where the 
nine teachers in the CAI/CMS project were interviewed; Data collected in the 
CAI/CMS observations included teacher respoixses to an interview instrument. 
Visitati on^ to CAI/CMS Cl asses. Observations were made by another evaluator in 
four regular SDR units with interviews of teachers being conducted on an 
instrument, Evaluator's gisitatlon Log.sheet . A copy of each of the observation 
Instruments is found in the Appendix. 

Major Findings 

Due to the fact that the 1985-86 SDR Program contained two treatment groups 
(regular instruction group and CAI/CMS group), data on enrollment/attendance 
and achievement testing are reported below in two waysi^ Thesa data are first 
presented for the overall program regardless of treatient group. The second 
presentation compares the two treatment groups in regard to 
enrollment/attendance data and achievement test datai 

interpreting the pretest-pOsttest achievement data, the reader should be 
aware of the pupil selection process. Previous nora-ref erenced i-tading 
achievement data and staff recommeridatlons were used to select and enroll 
pupils for the SDR program. To be eligible for the prograi (Objective 1.1) the 
pupil had to score at or below the 36th percentile on the selection test. Once 
the eligibility list was established, pupils were selected in order of their 
test scores with the lowest scoring pupils selected first. Following 
enrollment, pupils were pretested on the CTBS Reading Coipreheusion subtest. 
Level J Form Ui 

Pupil Census Information 

During the 1985-86 school year the SDR Program served 909 pupils. Of the 
909 pupils, 877 (96.5%) were ninth-grac. >rs ^ and 31 (3.4:^) were tenth graders 
and there was one eleventh grade pupil (0.1%). Of the 909 pupils, 559 (61.5%) 
attended the minimum number of days (107^ to meet. the 80% attendance criterion 
level contained in Objective 1.1. This was an Increase of 5;6% over last 
year s figure of 55.9%; A breakdown by grade level showed that 548 (62.5%) of 
the ninth-graders, and 11 (35.5%) of the tenth-graders met the attendance 
criterion. The one eleventh-grader did hot mike the attendance criterion. The 
overall attendance rate for the program (toi.al days of attendance divided by 
total days of enrollment) was 84.7%, as compared to 83.6% last year. The 
average number of days of enrollment and attendance for program pupils was 
117.9 and 99.8 respectively. The average dally membership was 799;6, which was 
an average of 53.3 pupils per teacher as compared to 47.4 pupils per teacher in 
last year's program. Table 1 contains the pupil attendance data. 
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Table 1 

Number of Pupils Served; _ Ave rages for Days of Enrpllraerit, Days of Attend 
Daily Membership and Hours of Instruction Per Week; and 
Pupils Attending 80% of D:ays 
Reported by Grade Level 















Average 






Pupils 
Served 












Grade 


Girls 


Boys 


Days of 
Enrollment 


Days of 
^ tendance 


Daily 
Membershio 


Hours of Instrnc 
per Pupil per We 


9 


877 


401 


476 


118;8 


100,6 


777.4 


3.6 


ib 


31 


10 


21 


92.7 


77.0 


21.4 


3.5 


ii 


i 


i 


0 


105.0 


100.0 


0.8 


3.8 


Total 


909 


412 


497 


117.9 


99.8 


799.6 


3.6 



B 
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The eyaluatibri sample cbnslsted of those pupils who met four criteria: 
attended 80% (107)_.6f the 13_4 program days^^ received both a pretest and a 
posttest with the CTBS^ scored at or below the 36th_ percentile on a selection 
test, and who were judged to be English speakings Of the 909 pupils served by 
thB program^ 467 (51.4%) were in the evaluation sample. 

Standardized Achievement Test Information 

The analysis of pretest-posttest achievement data provided minimuras, 
maximumis, averages or medians^ and differences for derived scores by grade 
level. The derived scores used in the analysis were percentiles, grade 
equivalents, and normal curve equivalents. No raw score data is presented 
because pupils took a different form of the test at pretest and pcsttest 
times. Therefore, pretest-pbsttest comparison of raw scores would be 
questionable. 

Table 2 contains pretest^posttest percentile data. The median percentile 
for the pretest was 12.5 at grade 10 and 30.5 at grade 9; The median 
percentile for the posttest was 9.3 at grade 10 and 15.1 at grade 9; These 
data indicate that neither grade approached a median percentile score of 36 at 
posttest time. Further analysis of pretest percentile di&tributions indicated 
that 42.6% of the ninth grade pupils scored above the 36th percentile on the 
pretest, even though they had previously qualified for the program on a 
selection test. Since the program serves mostly ninth grade, this represents 
42.0% of the overall evaluation sample. 

- _ Table 3 contains pretest-posttest grade equivalent data. The median grade 
equivalent iscore decreased from 8.0 to 7.8 at grade 9 and stayed the same (8;1) 
at grade 10. 

The_ presentation of achievement data thus far has included results from the 

analysis of percentiles arid grade equivalents . Both percentiles and grade 
equivalents provir^e comparative information but are not eqiiai units of 
measure; Caution is advised in draxd.rig conclusions about program impact from 
any of _ the scores above. Normal curve equivalents (NOE's) are generally 
considered to provide the^ruest iridicatipn of pupil growth in achievement, 
since they provide comparative information in _equal units of measurement. Data 
for normal curve equivalents are presented in Table 4. 

j_i Objective 1.1 states that the evaluatiDri sample would be composed of pupils 
wfeo scored below the 36th percentile oh the selection test and were in 
attendance 80% of the program's treatment period. In order to meet the 
attendance criterion the pupil had to attend at least 107 days of the seven 
month (134 days) treatment period. To achieve Objective 1.1 the average growth 
in_ reading achievement of pupils irt the evaluation sample had to be 1.0 NGE's 
for each month of the treatment period which is an average of 7.0 NCE's for the 
seven ii:onth program. 

overall NCE change for the program was -8.5 or an average of -1.2 NCE's 
for each of the seven months of the treatment period. Tl^ils negative change 
fell _cons_iderabiy short of the expected evaluation criterion of 1.0 NGE's 
gained for every month che pupils were iri the program. A negative change of 
-8.^7 NGE's or -1.2 NGE's per month occurred ^.n grade 9. In grade 10 there was . 
a j)ositive change of 1.2 NCE points, or 0.2 NGE's per month. However, the 
sample size at grade 10 was very small (seven pupils). 
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Table 2 

MinirauDij Maximum, Median, and Standard Deviation 
of the Pretest and Posttest Percentiles 
Reported by Grade Level 



Number 

grade of Pupils Hin^ 



9 
10 



460 
7 



Pretest 



Max. 



84 
28 



Median 
Percentile 



30.5 
12.5 



Standard 
Deviation 



le.9 
8.6 



Posttest 



Min; Max. 



99 
37 



Median 
Percentile 



15,1 
9.3 



Standarc 
Deviatic 



18.6 
15.7 



12 



13 
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Table 3 

Minimum, Maxiraum, Median and Standard Deviation 
of the Pretest and Posttest Grade Equivalents 
Reported by Grade Level 



PretesJ^ Posttest 

Median Median ' 

r ^ .^"'''ff ^^"^ Standard Grade Standar, 

Grade of Pupils Mln. Max. Equivalents Deviation Mln. Max. Equivalent Devlatioi 



9 460 4.2 12.9 8.0 1.7 4,0 12.9 7.8 

7 4.2 9.1 8.1 1.7 4.2 9.7 8.1 
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Table 4 



Minimur., Maxiraura, Average* and Standard Deviation of the 
Pretest and Posttest Normal Curve Equivalents (NCE) 
Reported by Grade Level 



— Post test 



- Pretest 

rr-.^. f'^^, „ ^^^^ standard Average" Standard Averag, 

Grade of Pupils M±n; Max. NCE Deviation Min. Max. NCE Deviation Chang, 



5 ^60 15.4 70.9 38.6 11.3 1.0 99.6 30.0 14.7 -8.7 

l.G 43.0 24.7 16.8 1.2 



10 7 i;0 37.7 23.5 11.7 

Total 467 38.4 11.4 



29.9 14.8 -8.5 



16 j7 
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It should be noted that NCE scores are based on percentiles, which compare 
the pupil's performance in relation to the general population^ No change in 
NCE score would indicate that pupils have progressed at their normal rate of 
growth over the school year. Even a _ small gain in percentile or NCE score 
^^^^ i^^icate that pupils have advanced over the school year at a greater rate 
than would be expected from their original position in relation to the general 
population. Table 5 contains data related to the changes in NCE scores for 
three ranges: (a) no imptrbyement _in NCE scores (OiG or less), (b) some 
improvement in NCE scores (0,1 to 6.9), and (c) substantial improvement in NCE 
scores (7.0 or more). The data indicate that 114 (24.4%) pupils iade gains in 
NCE scores. This means that lk.A% of the pupils in the evaluation sample 
progressed at a rate that was greater than normal for them. More specifically, 
65 (13.9%) made significant improvement, and 49 (10.5%) made some improvement in 
NCE scores, while 353 pupils (75.6%) of the evaluation saipie made no 
improvement, as evidenced by a gain of 0.0 or decrease in NCE score. In regard 
to grade level, five of seven C71.4%) tenth grade pupils showed progress, while 
109 of 460 (23.7%) of ninth grade pupils showed positive progress. 



it is posited that the apparent lack of pupil progress at the ninth grade 

level may have been due in part to three factors in the testing process, two 
occurring before or at the pretest and one occurring at the posttest. These 
factors are the following: 

li Prescheduling of classes for SDR. 

2. Outdated selection test scores. 

3. Problems in the administration of the customized 
posttest . 

The first two factors occurring before or at pretest are intimately 

connected. Pupils in the SDR prbgrani are plrescheduled in the spring of the 
previous year. This prescheduling takes place _ early in the spring before 
CLEAR, schooijd:de, or districtwide testing for __that _ year is completed. This 
means that some pupils may be scheduled into SDR classes based on selection 
scores j^ich are more than one year old. As noted earlier, 42.6% of the ninth 
grade pupils scored above the 36th percentile in the pretest, although all 
pupils had previousiy qualified for the program on a selection test. This may 
in fact be an artifact of the prescheduling and selection test process * 

The third factor which might have affected puiJil progress involved problems 
in t:he_ administration of the customizec posttest for grade 9. These problems 
were discovered during process evaluation at selected schools during the 
districtwide posttest. Generally, these problems were caused by improper 
scheduling and preparing for the posttest. The Examiner 's Manual for CTBS gave 



Table 5 



Change eategories for NCE Scores 
for Total SDR Program 

Pupils No Improvement^ Some Improvement Substantial imp rovemen*- 

in Sample (0.0 or less) (0.1 to 6.9) (7.0 or more) 

~ — — 

fiber of Pupils 460 351 46 63 

>f Pupils 76.3% 10.0% 13.7% 

i 10 

iber of Pupils 7 2 3 2 

Pupils 28.6% /-2.9% 28.6% 

.8 

iber of Pupils 467 353 49 65 

^ P"Pi^« 75.6% 10.5% 13.9% 
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Mariy^ if not all, of the posttest problems could be solved by more careful 
planning by building administrators and staff. The following are examples: the 
testing Sessions were held on Thursday and Friday because a field trip to the 
career centers on Tuesday was given priority over the testing, which was 
originally scheduled earlier in the week; Students were given all tests in one 
sitting with a short break between reading and mathematics tests. Directions 
were read over the loudspeaker to all ninth-grade homerooms in one high school. 

Obviously it Is hard to extrapoiate how many more problems would have 
surfaced if all schools had been observed during posttest week; however^ it is 
certain that process evaluation uncovered some possible reasons why pupil 
progress at the n.-'.nth grade ievei was not as much as anticipated. An interim 
report i Observations of Selected ii^lassraoms- JDurJLng^ mstrictw^ Testing , 
detc-.iling the results of the districtwide observation process evaluation, was 
sent, to an Assistant Superint-?:ndent of the €olumbus Public Schools, 

Program, teacher interviews during school visitation process evaluation 
indicated that teachers wore not satisfied with the large group testing at 
posttest time. They believed that proper testing procedures demanded the same 
testing conditions at both the pretest and the posttest. Program teachers also 
felt that there was the possibility of judging their pupils' performance as 
being deficient when in fact the problem wa? testing conditions. In the 
previous school year (1984-85), the posttest was administered by program 
teachers. Although the change scores for that year were also negative, the 
negative changes were less extreme than in the 1985-86 school year. 

Teacher perceptions of pupil progress , as measured by an item on the Pupil 
Census Form, _ suggested that there was more pupil progress than test scores 
indicated. Of the 909 pupils served by the program, teachers rated 299 (32.9%) 
as making much progress, 401_ (44.1%) as inaking some progress, 135 (14.9%) as 
leaking little progress, and 73 (8.0%) as making no progress. One Pupil Census 
Form was not counted in regard to this item due to multiple marking. 

Tables 6-10 present comparisons between the group of pupils receiving 
computer assisted instruction/computer management system (CAl/CMS) in reading 
and the group receiving the regular program instruction; As indicated in Table 
6, there were 569 pupils served by the CAl/CMS project and 1^40 pupils who 
received regular reading instruction. The CAi/ CMS group averaged slightly more 
days of attendance per pupil with an overall average of .100.2 days as compared 
to 99.1 days for the regular group. The average number of days attended was 
greater for grade 9 than for grade 10 in both the CAl/CMS group and in the 
regular group. In_ the CAl/CMS group 364 of the 569 pupils served (64.0%) met 
the program attendance criterion bv attending at- ^pffsst in? riavo Tts t-u^ 
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Table 6 

Number of Pupils Served, Averages for Days of Enrollment, Days of" Attendance, 
Daily Membership and Hours of Instruction Per Weeks and 
Pupils Attending 8Q% of Days Reported by Grade Level 
for Pupils Receiving Reading Instruction with Computers (CAI/CMS Group) 
and Pupils Receiving Reading instruction without Computers (Regular Group) 



Pupils 
B Served 


Girls 


Boys 


Days of 
Enrollment 


Days of 
Attendance 


Daily 
Membership 


Hrs. of Inst. 
Per Pupil Per Week 


mpiis 
Attending 
80^ of Days 


Group 
















555 
14 

b 


253 
8 
0 


302 
6 
6 


118.3 
85.1 


100.9 
75.6 


490.1 
8.9 


3.5 


359 


569 


261 


308 


117.5 


100.2 


499.0 


3i5 


364 


Group 
















322 
17 
1 


148 
2 
1 


174 
15 
0 


119.5 
99.0 
105.0 


100.2 
78.2 
100.0 


2P7.3 
12.6 
0.8 


3i6 
3.S 
3.8 


189 
6 

b 


340 


151 


189 


liSiS 


99.1 


300.6 


3.6 


195 
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Table 7 

Minlnum, Maxininnr, Median, _and Standard Deviation 
of the Pretest and Posttest Percentiles Reported by Grade Level 
for Pup:ils Receiving Reading instruction with Coraputsrs (CAI/CMt Group) 
and Pupils Receiving Reading instruction without Computers (Regular Group) 



Rumber 
of Pupils 



Pretests 



Median 

MJn. Max. Percentile 



Standard 
Deviation Min. 



Posttest 



Median Standard 
Max. Percentile Deviation 



301 
3 



5 

12 



84 
28 



30.5 
i6.0 



16.8 
9.2 



86 
37 



14.6 
9.0 



17.2 
18.9 



159 
4 



84 
18 



30.8 
8.0 



17.2 
7.7 



99 

11 



17.7 
12.5 



20.9 
15.8 
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grbu^.._ At grade 10 the median _percent±ie regressed from 16.0 to 9.0 in the 
CAI/CMS group but progressed fr-om 8.0 to 12.5 in the regular treatment group. 

_ Table .8 presents comparisons in.^ terms of median grade equivalent scores. 
Positive, charges occurred in both grades of the regular tr-atment group but not 
iri the CAI/CMS group. The median _grade equivalent score decreased from 8.0 to 
7.7 in grade 9 of the CAI/CMS group ,^ A positive change occurred in grade 9 of 
the regular treatment group where the_ median grade equivalent score increased 
from 8.0 to 8.1. In grade 10 ^ere the samples were smaller, the regular 
grpup/s mediaji grade equivalent scojre increased from 6.3 to 8.2^ wUile the 
CAI/CMS group's median grade equivalent score decreased from 8,3 to 7,9, 

As indicated earlier, NCE scores are generally considered to provide the 
most comparative information in equal unita of measurement. Data for the two 
groups in terms of NCE scores are presented_ in Table 9. The data indicate that 
the average NCE chanjge within the CAI/CMS group was -9.8 NCE points in grade 9^ 
where there were 301 pupils in the sample, and -7.5 NCE points in grade 10^ 
where there were 3 pupils in the sample. In the regular treatment group the 
159 pupils in grade 9 _had an average change of -6.5 NCE points, and the very 
small sample of A p^upils in grade 10 had an average gain of 7.8 NCE points. Of 
all the groups only the grade IG regular SDR group met the criterion of 
Objective 1.1 with a change of 7.8 NCE points, or i.i NCE points for each month 
of instruction. An overall comparison of the two treatment groups is obtained 
by examining the average NCE changes across grade levels. The average change 
for the CAI/CMS Jroup was -9.8 NCE points over the seven month treatment 
period. The regular treatment group did _some what better with an average change 
of -6.1 NCE points in the same treatment period. 



Table 10 compares the CAI/CMS and regular groups in regard to numbers and 
percents of pupils who evidenced no improvement, some Improvement, and 
substantial improvement! as previously defined. The data Indicate that 51 
pupils (31.3%) of the. regular group pupils, made positive gains in NCE scores, 
while 63 pupils _(2P.7%5 of CAI/CMS groups did so. Positive gains in the 
regular group included 34 pupils (20.9%) -v^o made substantial improvement arid 
17 pupils (10.4%) who made, some improvement. Positive gains in the CAI/CMS 
group^ included 31 pupils (10.2%) making substantial improvement, and 32 pupils 
(10.5%) making some improvement. 

Objective 2,1 stated that program personnel would be provided at least tw) 
inservice sessions arid that at least 80% of . the personnel attending each 
session would rate the ses??ion_as valuable in providing information that would 
assist them in carryirig out their program responsibilities. A total of five 
inservice meetings was provided by the Department of Federal and State 
Programs. Each program teacher was given the oppo_rtunity to attend either 
three or four of the meetirigs. On September 3, 1985, an orientation meeting 
ras held for all SDR teachers. A modified version of the General Inservice 
Evaluation Form was used for this meeting while ail of the other inservice 
meetings were assessed using the regular General Inservice Evaluation Form. 
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Table 8 



1 Minimum, Maximum, Median, and Standard deviation 
of the Pretest and Posttest Grade Equivalents Reported by Grade Level 
for Pupllls Receiving Reading Instruction with eomputers (eAi/CMS Group) 
and Pupils Receiving Reading Instruction vd.thout Compaters (Regular Group) 



Jrade 



Pretest 



Number 
of Pupils 



Min. 



Max. 



H/CHS Group 

9 301 

0 3 



4.2 
8.0 



12.9 
9.1 



Median 
Grade 
Equivalents 



8.0 
8.3 



Postte st 



Standard 
Deviation 



Median 

Grade Standard 
Max; Equivalent Deviation 



M±n; 



i;7 
0.6 



4;G 
4.2 



12.9 
9.7 



7.7 
7.9 



1.9 
2.8 



igular Group 

9 159 

0 4 



4.2 
4.2 



12.9 
8.6 



8.0 
6.3 



i;8 
2;i 



4;0 
5;0 



12.9 8.1 
9.7 8.2 



2.0 
2.1 



28 



27 
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Table 9 

Minimum^ Maximum^ Average, and Standard Deviation of the 

Pretest and Pbsttest Normal Curve Equivalents (NGE) Seported by Grade Level 

for Pupils Receiving Reading Instruction v^th eompaters (CAI/CMS Group) 
and Pupils Receiving Reading Instruction without Corapoters (Regular Group) 

Pretest jjs^gttest 

r A /^f'^ff Average Standard Average Standard Average 

Grade of Pupils Min. Max. nCE Deviation Min. Max. NCE Deviation , Change 



lyeMS Group 

9 301 15. ^ 70.9 38.9 U.O i.0 73.0 29.1 13.8 -9.8 

19 3 25.3 37.7 29.4 7.2 1.0 43.0 21.9 21.0 -7.5 



■otal 304 38.8 U.O 29.1 13.9 -9.8 



;qlar Gronp 

5 159 15.4 70.9 38.1 11.8 1.0 99.0 31.6 

*9 * I'O 30.7 19.1 13.3 6.7 43.0 26.8 



16.3 -6.5 
16.0 7.8 



otai 163 37.6 12.1 



31.5 16.2 -6.1 



28 

29 
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Change Categories for NCE Scores for Total SDR Program Reported by 
Grade Level for Pupils Receiving Reading Instruction with Computers 
(CAI/CMS Group) and Pupiis Receiving Reading Instruction 
without Computers (Regular Group) 





Pupils 
in Sample 


No improvement 
(0.0 or less) 




Some Improvement 
(0.1 to 6.9) 

- - 


Substantial Improvemc 
(7.0 or more) 


CAI/CMS Group 










ox. due y 










Number of Pupils 
% of Pupils 


301 


239 
79.4% 


10.6% 


30 

10.0% 


Grade 10 










Number of Pupils 
% of Pupils 


3 


2 

66.7% 


b 

0.0% 


1 

33.3% 












Number of Pupils 
% of Pupils 


304 


241 
79.3% 


32 

10.5% 


31 

10.2% 


Regular Group 










Grade 9 










Number of Pupils 
Z of Pupils 


159 


112 
70.4% 


14 
8.8% 


33 

20,8% 


Grade 10 










Number of Pupils 
% of Pupils 


4 


e 

0.0% 


3 

75.0% 


1 

:!5.o% 


Total 

Number of Pupils 
% of Pupils 


163 


112 
68.7% 


17 

10.4% 


34 

20.9% 
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Insery-lxre-fivaiuati on In f^ matla n 

Analyst ratings to individual inservice meetings indicated 

that 83i3% of the teachers attending the overall prbjgram meeting of September 3 
either agreed or strongly agreed that the program was worthwhile and would 
assist them in their program* Of the other four, inservice meetings^ three 
received favorable ratings by 100% of the participants. (inly one meeting 
received favorable ratings by less than 80% of the partic-lpants , at 77.8%. 
Table 11 contains a summary of the cdmbiried teacher ratings for all of the 
inservice programs. In this combined ratings 90. 2% of the participants agreed 
or strongly agreed that the information in the meetings would assist them in 
their program. Ratings were based on the following five-point scale: 

1 = Strongly Disagree 

2 = Disagree 

3 = Undecided 

4 = Agree 

5 = Strongly Agree 



Table 11 

Average Response and Percent of Response 
For Reactions to Inservice Statements 



Percent 



Statements- 



Number Average SA A 

Responding- -Response (5) (4) 



U 
(3) 



D SD 
(2; (1) 



li I think this was 
a very worthwhile 
meetings 

2i The information 
presented in the 
iLeeting will assist 
me in my program. 

10 There was time to ask 
questions pertaining 
to the presentation. 

4i Questions were 

answered adequately. 



41 

41 

41 

4b 



4.4 

4.4 

4.4 
4.4 



61.0 31.7 0.0 4.9 2.4 



56.1 34.1 4.9 2.4 2.4 



56.1 36.6 
57.5 35.0 



2.4 
2.5 



2.4 
2.5 



2.4 
2.5 



Open-ended conmients on the General Inservice Evaluation Form asked 
participants to ^comment about the riiost and least valuable parts of the meetings 
and about information they^uld like to have covered in future meetings. Only 
those open-ended comments \^ich were made by three or more plarticipants at any 
single session will be summarized here. However, the evaluation reports on 
individual sessions have been forwarded to the Department of State and Federal 
Programs and are available on request. 

In regard to_ th(^ m^ost valuable parts of inservice meetings, teachers liked 
the one giving lesson plan_ ideas, designs, and forms. For the question dealing 
with the least valuable part, a frequent "non-answer" was "None/unknown/not 



^4 

EVA£SRVCS/P5ld/RPTFSDR86 ^ ^ 

REVISED 01/14/87 



EKLC 



20 



appiicabiei" There were no suggestions for future meetings with a frequency of 
three or more; 

It is conciuded that Objecti^^^ 2,1 was technically not attained, because 
one of the five sessions ras rated as valuable in carrying but component 
responsibilities by less than tl^^^^ 8(3% of the participants. Each of 

the four remaining sessions ,Jhiowever, was favorably rated by well over 80% of 
the participants.^ When a combined rating of all five inservice sessions were 
computed, an over ail average of 90^.2% of the participants rated the inservice 
sessions as valuable in carrying out component responsibilities. 

School Visitation Infomaxion 

In addition to the types of data specified in the evaluation desighi 
process evaluation data were obtained by means of on-site visits. All CAI/CMS 
teachers and four of the regular SDR teachers were interviewed during the 
school year during visits by a project evaiuator. These visits occurred in the 
period from February 5 through March 20, 1986; 

A locally developed instrument, Vi^ttation to CAI/CMS -Classes , was used to 
give structure to the interviews with CAi/CMS teachers. The instrument dealt 
with general program concerns, as^ well as items specific to a CAI/CMS setting.. 
High school prescheduling was found to aiieviate many problems of selection and 
schediiling for the program. However, high pupil mobility in some schools 
tended to disrupt the planned schedules, and also necessitated some selection 
testing when teachers were already working with scheduled pupils. Only two of 
the nine CAI/CMS teachers were satisfied with evaluative testing procedures. 
The CTBS v/as criticised as a testing instrument on the grounds that the forms 
of the test used in pretest and posttest were not equivalent and that the test 
was too difficult (especially the posttest). it was also pointed out that the 
pretest should be given sooner, before instruction has already been in 
operation for several weeks. Lab space was rated as mediocre or less by about 
half the CAI/CMS teachers. It was also confirmed through observation that 
about half the high school GA1/(MS labs are confined to small rooms. 
Environmental noise did not appear to be a significant problem, but there was a 
wide divergence in teacher ratings of environmental temperature. In ore 
extreme case, the lab was hot in warm weather and cold in winter and had 
windows that wouldn't open. Most technical difficulties with computers were 
minor and had been resolved. Effectiveness of the computers for diagnosis was 
rated by teachers with an average rating of 3;9 on a five-point scale, while 
computer effectiveness for instruction received an average rating of 4.3. Most 
pupil time in CAI/CMS labs was approximately evenly divided among the following 
activities: working at the computer, direct instruction, and individual 
seatwork. 

In the regular SDR visitations, interview were i^eiitered around several 
open-ended questions. When program teachers were asked about record keeping, 
the interviewr found that they were current in their record keeping, but 
generally they did not use the Department of Federal and State Programs' Pupil 
Data _Sheet, The teachers wanted a simpler method which did not involve 
shuffling sheets of paper daily. Contact with classroom teachers ras limited 
because of different conference _scheduies and classrooms being located on 
different floors; however, program teachers tried to maintain communication 
through notes and staff meetings. When asked whether they had concerns about 
the program's selection procedures for target pupils, their comments generally 
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did Tibt deal with the testing mechanics and referral systems but dealt with 
scheduling problems __instead. The question on testing procedures elicited 
cbinmients about the CTBS being too hard, too long, or having poor format; Also, 
one comment indicated that the time: limits for the test did not match the 
school's time peribd_ schedule. All teachers felt evaluation feedback was 
adequate, timely, and usefuli however, negative comments abbot the CTBS were 
again expressed. Teachers interviewed claimed the need for more materials in 
their classrooms. The question^ about temperature elicited comments about rooms 
being "burning or freezing." They were generally satisfied with environmental 
tldise levels. Pupils spent most of their time in individual seatwork 
activities, followed by direct instruction from the teacher next and work at 
learning centers last. Other concerns were centered on pupils' mobility and 
ischeduling problems within their buildings. 

Cbst^Benefit Analysis Information 

The pirogram evaluation included one further analysis not in the original 
evaluation design: a cost-benefit analysis, comparing the CAl/GMS group and the 
regular group. This analysis is summarized in Table 12. Costs included in the 
analysis included teacher salaries and _ the • contract cost for Prescription 
Learning Laboratory Reading Labs. Normal supplies and incidental costs v;ere 
not known in regard to the two groups but were assumed to be evenly 
distributed. Any error of cost estimate resulting from unknown costs would 
probably be in the direction of underestimating the cost for the Regular group, 
since most iastructibnal materials for the CAI/CMS group were included in the 
Prescription Learning Laboratory contract costs . The cost-benefit ar^iysis 
indicatied that the cost per pupil was $284.57 more per pupil in the CAi/CMS 
group than in the regular treatment group when computed on average daily 
membership. However, the use of computers enabled CAI/CMS teachers to serve an 
average of 5.3 more pupils per teacher than in the Regular group (based on 
average daily membership). 

In the CAI/CMS group 64.0% of the _ pupils served attained the program's 
attendance criterion, compared to 57.4% of pupils In the regular treatment 
group who met the attendance criterion. The evaluation sample, which depends 
heavily on attaitiruent of the attendance criterion, was. comprised of 53^4% of 
all pupils served in the CAI/CMS group compared to 47.9% of all pupils served 
in the regular treatment group. As noted earlier^ there was a negative change 
in NCE scorei? in both groups. The average change fpr_ the regular group was 
-6.1, while the NCE change for the CAI/CMS group was -9.8. 

Summary/Recommendations 



The Secondary Developmental Reading Program is an individualized learning 

program designed to assist secondary pupils who are haying reading problems. 
During the 1985-86 school year, 15 project teachers working in 13 senior high 
schools served a total of 909 pupils in grades 9-11. 

The program had two objectives. Objective 1.1 stated that pupils who 

attended 80% of the seven month treatment period would show an average gain in 
reading of 1.0 NCE's for each month, which is an average gain of 7.0 NCE's 
overall (seven months x 1.0 NCE's). This objective was not attained. The 
program showed an overall negacive change of -8.5 NCE points for the seven 
month treatment period, or -1.2 NCE's per month. In grade 10, the NCE gain was 
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Table 12 

eost-Benef it Analysis for 1985-86 Secondary Developmental Reading Progra 

Comparing Group Receiving Computer Assisted Instruction/Computer 

Management System (CAI/CMS) and Group Receiving Regular Program Instructi 



Number 
of 

Teachers 



-Program-Cost 



Total 



Per 
Teacher 



Average 
Daily Membership 

In Per 
Program Teacher 



Cose 

Per 

Pupil 



Percent 

of 
Pupils 
Meeting 
Attendance 
Criterion 



Ratio of 
Sample 

_to 

Pupils 
Served 



Ave rage 
NCE 
Gain 



9 493,698.73 54,855.41 499.0 55.4 



989.38 



64.0% 



53.4% -9.8 



211,865.82 35,310.97 300.6 50.1 



704.81 



57.4% 



47.9% 



-6.1 
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1.2 NCE's for. the treatment period, or 6^2 NCE's per montii. The negative 
change in grade 9 was -8.7 NCE's for the treatment period, or -1.2 NCE's per 
month. 

Most _ pupils _ served by the program TCre _in ninth grade. Examination of 
pretest data indicated, that the average pretest NCE score fpr ninth ^rade was 
38. 6i arid that 42.6% of the ninth grade pupils scored above the 36th percentiie 
on the pretest. It was posited that prescheduling of ciiujses for SDR, often 
based bri selection test scores more than a year old, :may account for the 
disproportionate number of pupils scoring above^the 36th percentile on the 
pretest. Ninth grade program pupils were posttested along with other ninth 
grade pupils as part of Dlstrictwide Testing,_ of ten in iarge group settings. 
Process evaluation during Distrlctwlde Testing revealed many infractions of 
good testing procedures. Relatively high pretest scores^ cox;?led with less 
than optimum cbriditipris_ in the administration of the posttest, may in part 
account for apparent lack of progress by pr ^jram pupils i 

Teacjier perceptions of pupil progress^, as measured by an item on the Pupil 
Census Form, suggested that there was iore jpupii progress than test scores 
indicated. Of the 909 pupils served by the program, teachers rated 299 (32.9%) 
as makirig much jprogress^ 401 (44.1%) as making some _progress, 135 (14.9%) as 
making little progress ^ and 73 (8.0%) as making ho progress. 

Objective 2.1 stated that program personnel would be provided at least two 
irservice meetings arid that at least 80% of the _personnel attending each 
meeting would rate the meeting as very worthwhile in providing information that 
would assist them in carrying out their program responsibilities. There was a 
total of five iriservice meetings provided by the Department of Federal and 
State Programs. Each program teacher was given the opportunity to attend 
either three or four of the meetings* All bu_t^ one__of the five meetings were 
rated as very worthwhile in carrying out program _responsibilities by more than 
the requisite 80% of participants. This objective was technically not attained 
because brie bf the_ five sessions was rated as valuable in carrying but 
cbmpbnent resporisibilities by less than the requisite 80% of the participants. 

The CAI/CMS prbjecti was located in eight high schools. The computer 
assisted units served 56? pupils, while 340 pupils _we re served in the Regular 
group. Neither the CAI/CMS pro ject group nor the group receiving regular 
program instructibriattained.the achievement cri^ CAl/rMS group had 

a negative change of -9.8 NCE's in a seven month per period^ whi'e the Regular 
group had a negative change of -6.1 NCE's ^ The only subset of the program to 
attain the achievemerit criterion of 1.0 NCE per month_ of instruction ras grade 
10 bf the Regular group, which had an average gain of 7»8 NCE's for the 
treatment poriod, br 1.1 NCE's per month of instruction. However, only four 
pupils wiere in this group. 

A cost-benefit stud_y indicated that cost per pupil was greater and NCE 
gainis ismaller in the CAI/CMS group than in the Regular group. Based on average 
daily membership, the cost per pupil was $284.57 more in the CAI/CMS group than 
in the Regular group. The Regular group made a negative average NCE change of 
"^•^r "^il^ the CAI/CMS group made a negative average change of -9^8. However, 
CAl/CMS teachiers were able to serve an average of _5i3 more pupils per teacher 
than in the Regular groupj based on average daily membership. Attendance also 
\ms somewhat better in the CAI/CMS group than in the Regular group as judged by 
the percent bf pupils attaining the program's attendance criterion of attending 
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80% of the days in a seven-month treatment period. The percent of pupils 
attaining this attendance criterion was 64;0% in the CAI/CMS grdup^ as compared 
to 57.4% in the Regular groups 

This year's increase in attendance can l5e viet^d as a success indicator. 
In the bverall .program, the percent_ of_ pupils attending 80% of the_ treatment 
period was 61.5%, an increase _of 5.6%^ over last year's figure of 55.9%. The 
overall, attendance rate for the program (total days of attendance divided by 
total days of enrollment) was 54.7%, as compared to 83.6% for last year's 
program. 

Another success indicator was the increased number of pupils served. Based 
on Average Daily Membership, this year's program served 53.3 pupils per 
teacher^ as compared to 47.4 pupils per teacher - in last year's program. 
However, the increase in teacher load might, be examined as a possible factor in 
the disappointing test results, ^long_ with problems of pupil selection and 
testing conditions noted earlier in this report; 

During the 1985-86 school year, the Secondary Developmental Reading Program 
experienced problems in several areas; 

1. Pupil achievement ; In terms of NGE scores, 75.6% of the pupils 
in the sample showed no improvement ; 10; 5% showed some 
improvement but did not attain the achievement criterion of 1.0 
NCE per month; and 13.9% met the achievement criterion. 

2. CAI/CMS project ; The CAI/CMS project evidenced considerably less 
growth in achievement test scores than did the Regular SDR 
group* As_a result, the CAI/CMS group did not attain the degree 
of cost effectiveness that was expected* 

3- Pupil selection; Program teachers perceived inconsistencies 
beteweeh selection scores obtained from tests given ::he previous 
spring arid_ performance on the fall pretest; The two tests were 
different forms of _ the same test (CTBS) ._ This criticism seems to 
be substantiated by examination of pretest percentile 
distributions where 42.6% of the pupils scored above the 36th 
percentile. 

Since __the Secondary Developmental Reading Program is to be continued for 
the 1986-87 school year, consideration should be given to the following; 

1. Conduct the program in schools that vd;li work with program 
personnel to reduce scheduling problems and increase program 
attendance. 
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2. Selection of pupils for the program should be based on the most 
current test data. However, the professional judgment of 
classroom teachers should be given _ considerable, weight in the 
selection process. A system should be_ devised for obtaining 
teacher recommendations from this feeder middle schools, 

3m Continue to evaluate the CAI/CMS part of the program with an eye 
toward finding more effective methods of serving the high school 
pupil who is experiencing reading problems. Further expansion of 
the CAI/CMS project is not warranted at this time until greater 
ef f ectivenesis can be demonstrated, 

^» ^^"^i^^ selection procedures, cbrrelatlori of course content to 
system's C ourse of - Study , instructional methods ^ class size, and 
test_ content to determine why pupils are not showing desired 
growths 



5. School administrators and staff should take the responsibility of 
assuring an optimum testihg ehviroriment by_ not scheduling 
unsuitable activities during testing weeks and adjusting class 
schedules to accommodate the length of the tests, 

6, Conditions for the pretest and for the pbsttest should be as 
comparable as possible with all examiners trained to give the 
tests per instructions in the Examiners' Manuals, Pupils should 

tested in groups larger than recommended by the testing 

company i 

^* program should be reviewed extensively in regard to policies 

and procedures, selection, scheduling, attendance patterns^ test 
administration, and achievement test scores. The review should 
determine whether the model for the program should be continued 
in its present form, modified, or discontinued. 
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ERIC 



ECIA CHAPTER 1 
ORIEKTATION INSEHVICE IVttBATION FORM 
September 1985 



Fund: 

(Circae only 



PrograzF 

(circle only .fiofi) 



nj Chapter 1 (2) DPPF (3) General 
in) Other (specify) 



(i) AiK (2) Aides £3) CLEAR - Eleo, (K-5) 

(45 aEAR-Hiddle (5) HSCA (6) OND 

(7) SDR (8) Regular Teacher 

(9) Other (specify) — 



Circle the nufiber that indicates the extent ti> which you agree with statenents 1-4 1 in 
rating the overall day of inservice. 



1. I think this was a very worthwhile 
inservice* 

2. The information presented in this 
insenrice will assist me in toy 
prbgrazD* 

3« The^^e was time to ask questions 
pertaining to the presentations, 

4. Questions were answered adequately. 



Strdn^y Strong'.y 

Disagree Disagree Undecided ^.gcfie Agree 



2 
2 



3 
3 



4 
4 



Circle the number that Indicates how_ you would rate each of the following portions of 
today's inservice in regard to interest and usefulness of presentations. 



Poor ilalr Good 



Excellent 



Sgperlor 



6. Large Group Session 

a. Interest 

b. Usefulness 



2 
2 



4 
4 



5 
5 



7. Commercial Exhibits 

a^ Interest 

b. Usefulness 

8. Mini-session with main sp^^^aker 

a. Interest 

b. Usefulness 



2 
2 

2 

2 



4 
4 

4 
4 



5 
5 

5 
5 
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ib. 



Poor Fair SiSS^ 


Exeellent 


Suoerior 


ChaD^er 1 mini* session 








a« Interest 1 2 


3 




5 


b« Usefulness 1 2 


3 


4 


5 


c. Clarity of instructions 1 2 


3 


4 


5 


Svaiuation Presentation 








a. Interest 1 2 


3 


it 


5 


b. Usefulness 1 2 


3 


4 


5 


c. Clarity of instructions 1 2 


3 


4 


5 


What was the most valuable part of tbis meeting? 








What was the least valuable part of this meeting? 








What additional information or topics would you like 


to see 


covered in 


future 


meetings? 
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GENERAL INSERVICE EVALUATION FORM 



Ihservlce Topic :^ 

Present er( 3) : 

Date: 



Session: 



a. ED. 



(e.g., 03/05/86) 
or p>m> 



Circle only the program you are in: 



ECIA Chapter 1 Programs: 

(1) ADK 

(2) CLEAR-Eleoentary Cl^55 

(3) CLEAR-Elemehtary-CAI 

(4) CLEAR-Hiddle School (6-8) 
(5^ CLEAR-Hlddle School-CAI 

DPPP Programs: 

(6) SDR (9-10) 

(7) SDR-(3ftI 

(8) HSCA 

Other (Specify) 



circle the number that Indicates the extent to which you agree with statements 1-4. 

Strongly _ _ Strongly 

Agree Agree Ondeoided Disagree Disagree 

1 • I think this was a very worthwhile 

meeting. 5 4 3 2 1 

2. The ihforma.tioh presented in this 
meeting will assist me in my 
program • 5 

3* There was time to ask questions 

pertainihg to the presentation. 5 

4. Questions were answered 
adequately. 5 

5. What was the ^mst valuable part of this meeting?- 



4 
4 
4 



3 
3 
3 



6. What was the least valuable part of this meeting?^ 



7. What additional information or topics would you like to see covered in future 
meetings ? - 
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Visitation to CAI/GMS Glasses 



Type of school (check one): ^^pe of computer 

E lementary 
M iddle School 
H igh School 



Record Keeping 

Current Comments 

Student Data Card 

Add Forms 

Pupil Census Forms "^^"^^^""""""'"""""""^ 

Pupil Progress Evidence cf; ' 



Selection Procedures 

No Problem s Too Time Consuming 

Inadequate Test othe r 

Too Complicated . _ 

Schedulin g . _ _ 



^Apple 

^PET 

— ^Dolphin 
Other - 



Testing Procedures 
No Problem s 
inadequat e- 
Not Applicable 



Too Complicated 

Too Much Time 

Other 



Evaluation Feedback 
Adequate 



Problems 



Useful 



Facilities 
Space 

Materials 

Computer Effectiveness 

1. For Diagnosis 

2. For Instruction 

Environmental Temperature 
Environmental Noise Level 



Very Adequate 



Timely- 



5 




3 


2 


5 




3 


2 


5 




3 


2 


5 




3 


2 


5 _ 


i» 


3 


2 


Good 








5 


It 


3 




5 


i» 


3 


2 



Inadequate 



Very 



Poor 
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Computer Technical Difficulties 

Were the Problems 
J^q uency of pG Ourrenoe Resolved Satisfactorily 

Seldom 

o r N e v er Qeo aslbrial ly FrequerltJ^y Tes Uo Partially 
a. Minor dlfflcuitles 



b. Major difficulties 



What percent of a pupil *s time is typically used in each of the following 
activities? 

Jl £t the computer 

% direct instruction by teacher (individual or group) 

% at learning centers/work stations 

% ill individual seatwork (other than learning centers) 

% Other 

(These should all add up to 100$) 
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ECIA Chapter 1 and DPFP-SDH Programs 
Eraluator's Visitation Logsheet 



Scboca 



I I etm-Qem (1-5) 

I I aEAR-md (6-8) 
n SDR (9-10) 



Date 



Program Teacher 



Evaluator 



1« Record Keeping 

Student Data Sheet 
Personal Data 
Testing Data 
Attendance 
Parent Involvement 

Add Forms 

PupU Cehsris Forms 
Pupil Progress 



e^^-rent 
Yes \ No 
I 



Comments 



Evidence of: 



General Comments about Record Keeping: 



Communication With: 
Classroom Teacher 



3* Selection Procedures 
Ho Problems 



Inadequate Test^ 
Too ComtlicatecT 



Hew 

Oft en- 



Share 
Progress 



Plan 

Activitiea 



General Comments about Frequency of Contact wivh Classroom 
Teacher(s) : 



Gv-.neral Comments about Kinds of Communication with dass- 
rdom Teacher(s): 



Too Time Consuming^ 
Other 



General Comments about Selection Procedures: 



4» Testing Prbcedijres 
No Problems 
Inadequate 
Not Applicable 



General Comments about Testing Procedures 



Too Complicated, 
Too Much Time 
Other 
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5. Evaluation Feedback 

Adequate 

Problems 



General Comments about Evaluation Feedback:- 



General Comments about Facilities: 



7. Hat,rlal3 V.i^ id^uat, ^ _ iSdi,,it. 

General Comments about Facilities: 



^ ^ Vei^r Good Yery Poor 

8. Emrironmental Temperature 5 4 3 2 1 

General Comments about Environmental Temperature: 



Very Good Ve^ Poor 

9* Environmental Noise Level 5^321 

General Comments about Environmental temperature: 



10. What percent of a pupil* s time is typicall:v used in each of the following 
activities? 

f direct instruction by teacher (individual or group) 

i at learning centers/work stations 

^ indiviUual seatwork (other than learning centers) 
% Other 



(These should all add up to 100$5 



General Comments about Pupils* Activities? 



11. Other Concerns and Comments: 



